Skip to content

Optimize parsing microseconds and offset in calendar #14169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jan 10, 2025
Merged

Optimize parsing microseconds and offset in calendar #14169

merged 2 commits into from
Jan 10, 2025

Conversation

@dkuku
Copy link
Contributor

@dkuku dkuku commented Jan 10, 2025 •
edited
Loading

Benchmarked for worst case scenario - 6digits and offset
datetime = "2015-01-23 23:50:07.012345-02:30"

Name                                          ips        average  deviation         median         99th %
Calendar.IOISO.parse_naive_datetime        4.61 M      216.69 ns  ±7284.06%         180 ns         320 ns
Calendar.ISO.parse_naive_datetime          2.23 M      448.94 ns  ±3245.43%         341 ns         621 ns

Comparison: 
Calendar.IOISO.parse_naive_datetime        4.61 M
Calendar.ISO.parse_naive_datetime          2.23 M - 2.07x slower +232.25 ns

Extended statistics: 

Name                                        minimum        maximum    sample size                     mode
Calendar.IOISO.parse_naive_datetime          150 ns    19540150 ns         5.49 M                   180 ns
Calendar.ISO.parse_naive_datetime            280 ns    13080565 ns         3.28 M                   331 ns

Memory usage statistics:

Name                                   Memory usage
Calendar.IOISO.parse_naive_datetime           720 B
Calendar.ISO.parse_naive_datetime             888 B - 1.23x memory usage +168 B

**All measurements for memory usage were the same**

Profiling Calendar.IOISO.parse_naive_datetime with eprof...

Profile results of #PID<0.1092763.0>
#                                               CALLS     % TIME µS/CALL
Total                                              32 100.0    6    0.19
:erlang.list_to_integer/1                           1  0.00    0    0.00
Enum.member?/2                                      1  0.00    0    0.00
anonymous fn/0 in CalendarStringBench.run/0         1  0.00    0    0.00
Range.new/2                                         1  0.00    0    0.00
:erts_internal.list_to_integer/2                    1  0.00    0    0.00
Calendar.IOISO.valid_time?/4                        1  0.00    0    0.00
Calendar.IOISO.valid_date?/3                        1  0.00    0    0.00
Calendar.IOISO.scale_factor/1                       1  0.00    0    0.00
Calendar.IOISO.parse_offset/6                       1  0.00    0    0.00
Calendar.IOISO.parse_offset/1                       1  0.00    0    0.00
Calendar.IOISO.parse_naive_datetime_guarded/2       1  0.00    0    0.00
Calendar.IOISO.parse_naive_datetime/2               1  0.00    0    0.00
Calendar.IOISO.parse_naive_datetime/1               1  0.00    0    0.00
Calendar.IOISO.parse_formatted_naive_datetime/8     1  0.00    0    0.00
Calendar.IOISO.do_parse_naive_datetime/3            1  0.00    0    0.00
Calendar.IOISO.days_in_month_guarded/2              1  0.00    0    0.00
Calendar.IOISO.days_in_month/2                      1  0.00    0    0.00
Enumerable.impl_for/1                               1  0.00    0    0.00
Enumerable.impl_for!/1                              1  0.00    0    0.00
Enumerable.member?/2                                1  0.00    0    0.00
Enumerable.Range.member?/2                          1  0.00    0    0.00
:lists.reverse/1                                    1  0.00    0    0.00
:lists.reverse/2                                    1  0.00    0    0.00
Calendar.IOISO.parse_microsecond/3                  7 16.67    1    0.14
Calendar.IOISO.parse_microsecond/1                  1 16.67    1    1.00
:erlang.apply/2                                     1 66.67    4    4.00

Profile done over 26 matching functions

Profiling Calendar.ISO.parse_naive_datetime with eprof...

Profile results of #PID<0.1092765.0>
#                                             CALLS     % TIME µS/CALL
Total                                            48 100.0    8    0.17
Integer.parse/1                                   2  0.00    0    0.00
Integer.count_digits_nosign/3                     6  0.00    0    0.00
Integer.count_digits/2                            2  0.00    0    0.00
:erlang.split_binary/2                            2  0.00    0    0.00
:erlang.binary_to_integer/2                       2  0.00    0    0.00
:erlang.binary_to_integer/1                       1  0.00    0    0.00
:binary.copy/2                                    1  0.00    0    0.00
Enum.member?/2                                    1  0.00    0    0.00
anonymous fn/0 in CalendarStringBench.run/0       1  0.00    0    0.00
Range.new/2                                       1  0.00    0    0.00
:erts_internal.binary_to_integer/2                3  0.00    0    0.00
Calendar.ISO.valid_time?/4                        1  0.00    0    0.00
Calendar.ISO.parse_offset/4                       1  0.00    0    0.00
Calendar.ISO.parse_offset/1                       1  0.00    0    0.00
Calendar.ISO.parse_naive_datetime_guarded/2       1  0.00    0    0.00
Calendar.ISO.parse_naive_datetime/2               1  0.00    0    0.00
Calendar.ISO.parse_naive_datetime/1               1  0.00    0    0.00
Calendar.ISO.parse_formatted_naive_datetime/8     1  0.00    0    0.00
Calendar.ISO.do_parse_naive_datetime/3            1  0.00    0    0.00
Calendar.ISO.days_in_month_guarded/2              1  0.00    0    0.00
Calendar.ISO.days_in_month/2                      1  0.00    0    0.00
Enumerable.impl_for/1                             1  0.00    0    0.00
Enumerable.impl_for!/1                            1  0.00    0    0.00
Enumerable.member?/2                              1  0.00    0    0.00
Enumerable.Range.member?/2                        1  0.00    0    0.00
Integer.parse/2                                   2 12.50    1    0.50
Calendar.ISO.valid_date?/3                        1 12.50    1    1.00
Calendar.ISO.parse_microsecond/3                  7 12.50    1    0.14
:erlang.apply/2                                   1 25.00    2    2.00
Calendar.ISO.parse_microsecond/1                  1 37.50    3    3.00

Profile done over 30 matching functions

For 8 digit in miliseconds the last ones are dropped - old code had separate case clause:
datetime = "2015-01-23 23:50:07.01234567-02:30"

Name                                          ips        average  deviation         median         99th %
Calendar.IOISO.parse_naive_datetime        4.01 M      249.66 ns  ±8174.86%         190 ns         511 ns
Calendar.ISO.parse_naive_datetime          2.01 M      496.91 ns  ±2761.43%         371 ns         982 ns

Comparison: 
Calendar.IOISO.parse_naive_datetime        4.01 M
Calendar.ISO.parse_naive_datetime          2.01 M - 1.99x slower +247.25 ns

Extended statistics: 

Name                                        minimum        maximum    sample size                     mode
Calendar.IOISO.parse_naive_datetime          150 ns    25745755 ns         4.78 M                   180 ns
Calendar.ISO.parse_naive_datetime            300 ns    12342405 ns         2.94 M                   361 ns

Memory usage statistics:

Name                                   Memory usage
Calendar.IOISO.parse_naive_datetime           720 B
Calendar.ISO.parse_naive_datetime             872 B - 1.21x memory usage +152 B

**All measurements for memory usage were the same**

Profiling Calendar.IOISO.parse_naive_datetime with eprof...

Profile results of #PID<0.931729.0>
#                                               CALLS     % TIME µS/CALL
Total                                              34 100.0    6    0.18
:erlang.list_to_integer/1                           1  0.00    0    0.00
Enum.member?/2                                      1  0.00    0    0.00
anonymous fn/0 in CalendarStringBench.run/0         1  0.00    0    0.00
Range.new/2                                         1  0.00    0    0.00
:erts_internal.list_to_integer/2                    1  0.00    0    0.00
Calendar.IOISO.valid_time?/4                        1  0.00    0    0.00
Calendar.IOISO.scale_factor/1                       1  0.00    0    0.00
Calendar.IOISO.parse_offset/6                       1  0.00    0    0.00
Calendar.IOISO.parse_offset/1                       1  0.00    0    0.00
Calendar.IOISO.parse_naive_datetime_guarded/2       1  0.00    0    0.00
Calendar.IOISO.parse_naive_datetime/2               1  0.00    0    0.00
Calendar.IOISO.parse_naive_datetime/1               1  0.00    0    0.00
Calendar.IOISO.parse_microsecond/1                  1  0.00    0    0.00
Calendar.IOISO.parse_formatted_naive_datetime/8     1  0.00    0    0.00
Calendar.IOISO.days_in_month_guarded/2              1  0.00    0    0.00
Calendar.IOISO.days_in_month/2                      1  0.00    0    0.00
Enumerable.impl_for/1                               1  0.00    0    0.00
Enumerable.impl_for!/1                              1  0.00    0    0.00
Enumerable.member?/2                                1  0.00    0    0.00
Enumerable.Range.member?/2                          1  0.00    0    0.00
:lists.reverse/1                                    1  0.00    0    0.00
:lists.reverse/2                                    1  0.00    0    0.00
Calendar.IOISO.valid_date?/3                        1 16.67    1    1.00
Calendar.IOISO.parse_microsecond/3                  9 16.67    1    0.11
Calendar.IOISO.do_parse_naive_datetime/3            1 16.67    1    1.00
:erlang.apply/2                                     1 50.00    3    3.00

Profile done over 26 matching functions

Profiling Calendar.ISO.parse_naive_datetime with eprof...

Profile results of #PID<0.931731.0>
#                                             CALLS     % TIME µS/CALL
Total                                            49 100.0   11    0.22
Integer.parse/1                                   2  0.00    0    0.00
Integer.count_digits_nosign/3                     6  0.00    0    0.00
Integer.count_digits/2                            2  0.00    0    0.00
:erlang.split_binary/2                            2  0.00    0    0.00
:erlang.binary_to_integer/2                       2  0.00    0    0.00
:erlang.binary_to_integer/1                       1  0.00    0    0.00
Enum.member?/2                                    1  0.00    0    0.00
anonymous fn/0 in CalendarStringBench.run/0       1  0.00    0    0.00
Range.new/2                                       1  0.00    0    0.00
:erts_internal.binary_to_integer/2                3  0.00    0    0.00
Calendar.ISO.valid_time?/4                        1  0.00    0    0.00
Calendar.ISO.parse_offset/4                       1  0.00    0    0.00
Calendar.ISO.parse_offset/1                       1  0.00    0    0.00
Calendar.ISO.parse_naive_datetime_guarded/2       1  0.00    0    0.00
Calendar.ISO.parse_naive_datetime/2               1  0.00    0    0.00
Calendar.ISO.parse_naive_datetime/1               1  0.00    0    0.00
Calendar.ISO.parse_formatted_naive_datetime/8     1  0.00    0    0.00
Calendar.ISO.days_in_month_guarded/2              1  0.00    0    0.00
Calendar.ISO.days_in_month/2                      1  0.00    0    0.00
Enumerable.impl_for/1                             1  0.00    0    0.00
Enumerable.impl_for!/1                            1  0.00    0    0.00
Enumerable.member?/2                              1  0.00    0    0.00
Enumerable.Range.member?/2                        1  0.00    0    0.00
Integer.parse/2                                   2  9.09    1    0.50
Calendar.ISO.valid_date?/3                        1  9.09    1    1.00
Calendar.ISO.parse_microsecond/3                  9  9.09    1    0.11
Calendar.ISO.do_parse_naive_datetime/3            1  9.09    1    1.00
:erlang.apply/2                                   1 18.18    2    2.00
Calendar.ISO.parse_microsecond/1                  1 45.45    5    5.00

Profile done over 29 matching functions

dkuku
dkuku commented Jan 10, 2025
View reviewed changes
lib/elixir/lib/calendar/iso.ex

defp parse_microsecond(<<head, tail::binary>>, 6, acc) when head in ?0..?9,
do: parse_microsecond(tail, 6, acc)

Copy link
Contributor Author

@dkuku dkuku Jan 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when there is more than 6 digits of milliseconds we just drop the head and continue.
It does not require special handling in the caller

@josevalim josevalim merged commit 7589754 into elixir-lang:main Jan 10, 2025
8 of 9 checks passed
@josevalim
Copy link
Member

josevalim commented Jan 10, 2025

💚 💙 💜 💛 ❤️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@dkuku @josevalim